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FOREWORD 


This  research  was  performed  in  support  of  the  Office  of 
Naval  Research  Organizational  Effectiveness  Research  Program.  The 
research  is  part  of  a larger  effort  in  which  a model  for  enhancing 
change  will  be  applied  to  the  introduction  of  a major  new  submarine 
sonar  system. 


J.  J.  CLARK1N 
COMMANDING  OFFICER 


SUMMARY 


Problem 


Misuse  of  new  hardware  systems  has  been  a recurrent  problem  in  the  Navy.  Parallels 
in  industry  and  education  can  be  noted,  and  the  use  of  a Change  Advocate  has  proven  help- 
ful in  some  civilian  settings.  Research  results  have  shown  that  specific  negative  attitudes  of 
users  are  a major  factor  in  the  misuse  of  a new  system.  Training  Navy  Change  Advocates 
will  require  the  development  of  an  effective  means  of  making  these  experienced  technicians 
aware  of  the  existence  and  effect  of  negative  technicians’  attitudes  on  the  utilization  of  new 
systems. 


Objective 

The  objective  of  this  research  was  to  investigate  a practical  means  of  selectively 
influencing  experienced  technicians’  attitudes  about  the  implementation  of  new  systems. 
Specifically,  the  goal  W3s  to  enhance  experienced  technicians’  awareness  of  the  existence 
and  adverse  effects  of  their  negative  attitudes,  while  neither  discrediting  the  existence  of 
other  causitive  factors  which  technicians  correctly  recognize,  nor  eroding  expectations 
of  new  systems. 


Approach 


Forty-nine  fleet-experienced  technicians  responded  to  a questionnaire  both  before 
and  after  being  exposed  to  one  of  three  experimental  conditions.  The  main  experimental 
manipulation  was  the  presentation  of  objective  evidence  in  the  form  of  records  of  ship- 
board observations  of  misuse  and  nonuse  of  the  ASROC  system  during  its  introduction 
to  the  Fleet. 


Results 

The  results  indicate  that  technicians  initially  endorsed  external  causes  of  system 
misuse  while  rejecting  internal  causes  such  as  their  own  attitudes.  However,  following 
exposure  to  ASROC  observations,  technicians  ( I ) more  readily  acknowledged  the  effects 
of  their  own  attitudes  as  well  as  external  factors,  (2)  maintained  positive  expectations 
of  new  systems  (with  some  decrement),  and  (3)  were  less  positive  in  their  evaluation  of  the 
effectiveness  of  Navy  hardware  systems.  A defense  statement,  which  was  designed  to 
temper  possible  overreaction  to  the  ASROC  observations,  produced  virtually  no  effect. 


vii 


Recommendations 


This  study  was  performed  to  provide  input  to  later  stages  of  a broader  research 
effort.  At  this  time,  the  results  are  not  interpreted  to  indicate  any  recommendations  for 
the  operational  Navy.  The  findings  have  been  considered  in  the  construction  of  a Change 
Advocate  training  program. 
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INTRODUCTION 


Background 

This  report  describes  the  second  study  of  a larger  research  effort,  which  will 
assess  the  effect  of  a Change  Advocate  role  and  a change  model  in  the  introduction  of  a 
new  computerized  system  to  the  Fleet.  This  larger  effort  was  undertaken  because  major  new 
systems  often  are  only  partially  used,  misused,  or,  with  respect  to  some  functions,  not  used 
for  years  after  their  introduction  to  the  Fleet.  Cases  of  such  misuse  have  been  documented 
(Mecherikoff  & Mackic,  1970).  The  evidence  indicates  that  many  factors  contribute  to  this 
problem.  A chief  factor  is  negative  specific  attitudes  held  by  technicians  toward  specific 
system  components  or  functions. 

In  the  first  study  in  this  research  effort  (Abrams,  Sheposh,  & Licht,  1974),  the  atti- 
tudes of  Navy  technician  team  members  toward  a technician  within  their  team  who  could 
potentially  fill  the  Change  Advocate  role  were  surveyed.  In  that  study,  the  role  was  identi- 
fied as  System  Specialist,  and  was  loosely  defined  as  follows: 

“The  System  Specialist  can  be  the  LPO  or  any  experienced  second  class  petty 

officer  or  higher.  In  addition  to  possessing  the  typical  job  skills  and  knowledges, 

he  should  have  the  ability  to  do  the  following  types  of  things: 

a.  Keep  up  to  date  in  his  knowledge  of  the  new  equipment. 

b.  Show  enthusiasm  about  his  role  as  System  Specialist  — encourage  others  to 
learn  about  the  system. 

c.  Communicate  with  people  in  supporting  activities  with  whom  he  will  discuss 
technical  problems. 

d.  Work  with  the  team  and  with  officers  aboard  the  ship  in  working  out  periodic 
problems  that  occur  with  almost  every  new  equipment. 

e.  Receive,  keep  track  of,  and  encourage  the  use  of  various  shipboard  training 
materials  which  will  be  sent  to  the  ship.” 

Results  revealed  that  the  experienced  technicians  recognized  the  need  for  a System 
Specialist  or  Change  Advocate  role,  and  that  technician  teams  aboard  surface  ships  and 
submarines  included  qualified  personnel  who  desired  this  role. 

The  present  study  attacks  a major  problem  that  might  be  encountered  in  training 
technicians  designated  to  fulfill  the  Change  Advocate  role. 


Problem 


Most  experienced  technicians  are  aware  of  the  lag  in  the  utilization  of  a new  system, 
and  most  are  aware  of  many  of  the  probable  causes.  It  is  suspected,  however,  that  they  tend 
to  acknowledge  the  existence  of  external  causes  (e.g.,  poor  equipment  design,  poor  docu- 
mentation, etc.)  of  misuse  or  nonuse,  but  they  are  reluctant  to  accept  the  possibility  that  some 
of  their  own  attitudes  may  be  a major  part  of  the  problem.  It  is  considered  highly  unlikely  that 
a technician  who  is  designated  to  perform  a change  advocate  function  will  recognize  either 


the  existence  or  the  effect  of  specific  negative  attitudes  toward  the  new  system.  Yet,  an 
important  part  of  his  role  will  be  to  deal  with  such  attitudes.  Success  in  dealing  with  this 
problem  is  necessary  if  a technician  team  is  to  use  a new  hardware  system  more  objectively 
and  effectively. 

This  study  investigated  ways  of  making  experienced  technicians  aware  of  the  exis- 
tence and  adverse  effects  of  negative  attitudes  toward  system  components  or  functions, 
while  not  discrediting  the  importance  of  other  causative  factors  which  they  correctly 
recognize. 
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PROCEDURES 


Subjects 

Forty-nine  fleet-experienced  technicians  served  as  subjects.  The  mean  amount  of 
active  duty  time  for  this  group  was  1 1.4  years.  Table  1 presents  a breakdown  of  subjects  by 
rating  and  rate  level. 

TABLE  1.  DISTRIBUTION  OF  SAMPLE  BY 
RATING  AND  RATE  LEVEL. 

Rate  Rating 


Level 

ET 

EW 

FT 

OS 

ST 

TOTAL 

E-8 

— 

— 

1 

— 

1 

2 

E-7 

4 

2 

— 

2 

4 

12 

E-6 

9 

3 

— 

6 

7 

25 

E-5 

7 

- 

- 

2 

1 

10 

TOTAL 

49 

The  sample  was  restricted  to  experienced  technicians,  second-class  petty  officers 
and  above,  to  assure  that  all  had  some  fleet  experience  with  new  hardware  systems.  All 
subjects  were  instructors  from  three  technical  schools  in  the  San  Diego  area.  Subjects  from 
each  school  were  randomly  assigned  to  each  of  three  experimental  conditions. 


Experimental  Design 

This  study  incorporated  the  following  elements:  (1)  directing  subjects’  attention  to 
new  hardware  systems,  (2)  assessing  subjects’  attitudes  and  opinions  prior  to  experimental 
conditions,  (pretest),  (3)  presenting  information  in  one  or  more  experimental  conditions, 
and  (4)  reassessing  subjects’  attitudes  and  opinions  to  observe  the  effects  of  experimental 
conditions  (posttest).  Subjects  were  divided  into  three  experimental  groups.  These  groups 
were  employed  in  order  to  observe  the  relative  effects  of  presenting  two  different  types  and 
sequences  of  information. 

Group  I (known  as  the  Control  Group)  received  only  a set  of  cards  providing 
shipboard  observations  on  the  ASROC  system.  Group  2 (Defense  First  Group)  was  pre- 
sented a written  defense  statement  before  the  ASROC  observations,  and  Group  3 (Defense 
Last  Group)  received  the  defense  statement  after  the  ASROC  observations. 

The  experiment  was  conducted  over  a 2-day  period,  and  required  about  3 hours  per 
group.  The  first  day  was  devoted  to  setting  the  stage  by  describing  three  new  systems,  and 
administering  pretests  to  all  groups.  The  second  day  was  devoted  to  presenting  information, 
administering  posttests  to  obtain  postmeasures,  and  debriefing.  This  was  done  in  part  to 
reduce  or  eliminate  the  subject’s  memory  of  his  pretest  response  on  his  posttest  response. 
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1 1 lose  procedures  are  diagrammed  in  Figure  ! and  described  below: 


1.  Presentation  on  Three  New  Systems  — This  presentation  was  designed  to 
create  a setting  in  which  to  measure  opinions  and  attitudes  about  new  systems.  The  presen- 
tation referred  to  the  AN/BQQ-5  Sonar  System,  the  S3A  Air  ASW  System,  and  the  Noise 
Vibration  Monitor  Analyzer  (NVMA).  Each  new  system  was  briefly  described  in  terms  of: 

( 1 ) why  it  was  developed  (c.g.  what  were  the  compelling  tactical  or  operational  reasons), 

(2)  what  features  it  had  for  achieving  its  purpose,  and  (3)  its  present  status  relative  to  being 
operational  in  the  Fleet.  The  presentation  was  strongly  positive  in  the  sense  that:  (1)  assump- 
tions regarding  the  need  for  and  the  ability  of  the  system  to  meet  the  need  were  not  ques- 
tioned, and  (2)  potential  problems  were  not  explored. 

2.  Questionnaire  — Pretest  Administration  — The  questionnaire  consisted  of  two 
sets  of  items  comprising  four  types  of  questions.  The  first  set,  consisting  of  three  questions, 
assessed  subjects’  expectations  of  the  three  systems  described  in  the  presentation.  The  second 
set  consisted  of  18  items.  Four  of  them  referred  to  the  sufficiency  and  utilization  of  Navy 
hardware  systems  in  general  and  13  items  measured  the  degree  to  which  technicians  attrib- 
uted various  possible  causes  (e.g.,  lack  of  training,  poor  design,  negative  attitudes  to  problems 
encountered  with  new  systems).  All  of  the  above  questionnaire  items  were  rated  on  a 60- 
point  scale.  The  final  item  of  the  second  set  required  subjects  to  rank  order  seven  factors 
which  are  generally  considered  to  be  related  to  misuse  or  nonuse  of  new  systems.  The  ques- 
tionnaire is  presented  in  Appendix  A. 

3.  ASROC  Observations  — Shipboard  observations  of  the  misuse  and  nonuse  of 
the  ASROC  system  during  destroyer  exercises  were  obtained.  These  observations  had  been 
obtained  shortly  after  ASROC  was  introduced  to  the  fleet,  and  they  were  considered  to  be 
adequately  documented  so  that  technicians  would  generally  accept  them  as  valid  observa- 
tions of  their  work  environment.  With  reference  to  the  literature  on  persuasion,  the  obser- 
vations were  considered  to  be  statements  of  evidence  (McC'roskey,  1972).  For  this  study, 

a sample  of  33  observations  was  typed  on  5”  X 7”  cards.  Each  card  presented  the  following 
information:  (1)  the  ship  upon  which  the  observation  was  made,  (2)  the  situation  in  which 
the  observation  was  made,  (3)  the  system  component  which  was  misused  or  not  used, 

(4)  the  most  applicable  reason  category  (c.g.,  training,  design  of  equipment,  attitudes,  docu- 
mentation), and  (5)  the  probable  underlying  causes  (e.g.,  not  enough  on-the-job  training, 
controls  difficult  to  operate).  Specific  attitudes  toward  new  system  components  were  found 
to  be  the  most  frequent  underlying  causes  of  misuse  or  nonuse  both  in  the  ASROC  obser- 
vations and  22  of  the  33  observations  selected  for  use  in  this  study  identified  specific  atti- 
tudes among  the  underlying  causes.  Since  it  was  anticipated  that  technicians  were  unaware 
of  the  relationship  between  attitudes  and  nonuse,  it  was  speculated  that  evidence  presented 
in  this  form  would  be  effective  in  enhancing  awareness  (McCroskey,  1 972).  To  assure  that 
subjects  attended  to  each  card  as  they  read  it,  they  were  required  to  summarize  the  infor- 
mation it  contained  on  a record  sheet  (see  Appendix  B). 

4.  Written  Defense  and  Quiz  - The  written  defnese  was  developed  to  provide 
subjects  with  a less  threatening  context  in  which  to  view  the  relationship  between  specific 
technician  attitudes  and  the  misuse  or  nonuse  of  hardware  systems.  Four  major  causes  of 
misuse  were  reviewed:  equipment  design,  training,  maintenance,  and  attitudes.  An  attempt 
was  made  to  provide  greater  understanding  of  the  origins  of  each  cause  by  explicitly  identi- 
fying them  and  discussing  them.  To  determine  whether  subjects  attended  to  the  major 
points  as  they  read  this  statement,  they  were  given  an  “open  book”  quiz  after  they  finished 
it.  The  defense  statement  and  the  quiz  are  found  in  Appendix  C. 
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DAY  1 


DAY  2 


GROUP  1 
(CONTROL) 


GROUP  2 

(DEFENSE  FIRST)! 


GROUP  3 
(DEFENSE  LAST) 


PRESENTATIONS 
ON  THREE  NEW 
SYSTEMS 


Figure  1 . Elements  of  experimental  design. 


5.  Questionnaire  Posttest  Administration.  The  posttest  questionnaire  was 
identical  to  that  of  the  pretest. 

6.  Debrief  Questionnaire  — The  debrief  questionnaire  (See  Appendix  D)  provided 
each  subject  with  an  opportunity  to  report  what  had  occurred  from  his  perspective.  It 
included  six  direct  yes-no  type  questions  and  three  open-ended  questions.  Following  the 
completion  of  the  debrief  questionnaire,  there  was  a group  discussion  of  reactions  to  the 
experiment. 


Rationale 


In  the  general  research  program,  of  which  this  study  is  a part,  a change  model 
( Rogers,  1962;  Havelock,  1973)  is  being  used  as  a basic  framework  in  which  the  Change 
Advocate  will  operate.  In  this  model  the  first  stage  of  the  change  process  is  an  awareness 
stage,  in  which  the  recipient  of  the  change  becomes  aware  of  the  change  and,  particularly, 
its  effect  on  him.  It  is  important  that  during  this  stage  the  Change  Advocate  and  his  team: 

( I ) have  a realistic  picture  of  the  potential  problems  which  lie  ahead  as  a consequence  of  the 
new  system,  and  (2)  be  open-minded  in  considering  various  reasons  for  misuse. 

All  of  the  Change  Advocates  and  many  of  the  team  members  will  be  experienced 
technicians  who  will  enter  this  initial  stage  with  beliefs  based  on  a combination  of  their  own 
experiences  and  truisms  which  are  shared  by  technicians.  It  is  anticipated  that  these  beliefs, 
though  accurate  in  many  ways,  will  need  to  be  altered  during  the  awareness  stage. 

In  designing  this  study,  it  was  anticipated  that  technicians:  ( 1 ) would  have  a generally 
positive  expectation  of  new  systems,  (2)  would  identify  those  causes  which  were  technical, 
fairly  visible,  and  external  to  their  team  as  the  primary  sources  of  problems  with  new  sys- 
tems, and  (3)  would  less  readily  accept  causes  which  ascribed  fault  to  the  technicians’  team. 
The  latter  two  expectations  arc  consistent  with  the  conclusions  from  research  regarding 
perceptions  of  causes  of  behavior  (Jones,  E.,  & Nesbitt,  R.,  1971).  These  researchers  con- 
tend that  when  people  seek  to  explain  their  own  behavior  they  are  inclined  to  give  greater 
weight  to  environmental  causes  as  opposed  to  internal  causes. 

Major  concern  in  this  study  was  focused  on  enhancing  technicians’  awareness  of  both 
internal  and  external  causes  of  implementation  problems.  Many  of  the  problems  that  arise 
with  new  systems  are  technical,  quite  obvious,  and  not  the  fault  of  the  technician  team. 

For  example,  training  programs  and  materials  and  operator  and  maintenance  documentation 
have  often  not  been  adequate  for  early  users  of  a new  system.  Thus,  most  of  the  technicians’ 
beliefs  in  this  regard  are  well  grounded.  There  is,  however,  considerable  evidence  that  specific 
technician  attitudes  also  play  a role  in  the  misuse  or  nonuse  of  individual  modes,  displays, 
or  subsystems  of  new  hardware  systems.  Since  attitudes  are  internal  to  the  technicians,  it 
was  anticipated  that  technicians  would  underestimate  their  effects.  Thus,  one  primary  func- 
tion of  the  experimental  communication  was  to  increase  technicians’  acceptance  of  the 
effects  of  attitudes. 

Regarding  the  assumed  positive  expectations  of  new  systems,  it  was  felt  that  a real- 
istic positive  expectation  of  the  new  system  was  desirable.  Positive  expectations  which  were 
not  tempered  by  a realistic  assessment  of  potential  problems,  have  often  led  to  negative  over- 
reactions  when  predictable  problems  have  occurred. 

The  complex  question  which  was  investigated  in  this  study  might  be  worded,  “What 
type  of  communication  will  be  effective  in  enhancing  technicians’  awareness  of  their  own 
potential  negative  attitudes,  while  neither  altering  their  realistic  recognition  of  a variety  of 
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technical  problems  nor  destroying  realistic  positive  expectations  of  the  new  system?”  It 
was  decided  that  a prime  concern  would  be  to  present  a message  which  the  audience  would 
consider  good  supportive  evidence.  This  judgment  was  based  upon  the  results  of  research 
in  persuasion  (McCroskey,  1972;  McGuire,  1969),  Additionally,  the  nature  of  the  interac- 
tions between  the  researcher  and  the  Change  Advocate  trainee  and,  ultimately,  between  the 
Change  Advocate  and  his  team  ruled  out  techniques  such  as  fear  arousal  or  an  indirect  mes- 
sage; the  best  message  would  be  factual  and  direct. 

The  present  research  focused  on  an  initial  message  that  could  be  used  to  make 
technicians  recognize  that  negative  technician  attitudes  retard  the  implementation  of  a new 
hardware  system.  The  ASROC  observations  were  viewed  as  being  a highly  valid  type  of  new 
evidence,  since  they  documented  occasions  of  misuse  or  nonuse  of  a mqjor  hardware  sys- 
tem in  the  operational  setting,  and  the  data  had  not  been  easily  available  to  technicians. 

One  concern  in  presenting  the  ASROC  observations  was  that  the  effect  of  the  33  observed 
problems  might  generalize  beyond  the  goal  of  documenting  the  existence  and  negative 
effects  of  negative  technician  attitudes.  For  example,  research  findings  (McGuire,  1969; 
Wyer  & Goldberg,  1970)  have  shown  that  attitude  change  toward  one  issue  can  cause  indi- 
rect change  toward  related  issues  in  order  to  bring  about  greater  cognitive  consistency.  Thus, 
technicians  might  be  swayed  to  believe  that  attitudinal  effects  are  so  great  that  other  actual 
causes  of  misuse  are  of  minimal  importance.  Further,  even  if  cognitive  consistency  is  not  a 
major  factor,  the  mere  recitation  of  misuses  and  nonuses  of  a new  system  might  dampen  the 
technician’s  positive  anticipation  of  other  new  systems.  Also,  the  prevalence  of  reported 
problems  due  to  negative  specific  attitudes  might  distort  the  technician’s  realistic  evaluations 
of  other  sources  of  misuse  or  nonuse. 

To  prevent  the  observations  from  producing  possible  undesired  effects,  it  was 
decided  to  prepare  the  defense  statement,  or  immunization,  which  might  serve  to  defend 
against  such  results.  Since  it  was  not  known  whether  the  statement  would  be  more  effective 
if  it  preceded  or  followed  the  ASROC  observation  presentation,  the  study  was  designed  to 
determine  empirically  if  there  was  any  difference. 

The  following  questions  were  of  primary  concern  in  this  study: 

1.  Will  technicians  show  a greater  recognition  of  the  effect  of  specific  negative 
attitudes  upon  the  utilization  of  new  systems  following  exposure  to  the  ASROC 
observations? 

2.  Will  technicians  maintain  their  initial  position  regarding  external  causes  of 
misuse  following  exposure  to  the  ASROC  observations? 

3.  Will  technicians  have  less  positive  expectations  of  new  systems  following 
exposures  to  the  ASROC  observations? 

4.  Will  technicians  have  less  positive  overall  evaluations  regarding  the  effectiveness 
and  utilization  of  new  and  operational  systems  following  exposure  to  the  ASROC 

obse  rvations? 

5.  Will  the  statement  of  defense  alter  any  of  the  effects  of  the  ASROC  observa- 
tions, and  if  so,  will  the  sequence  of  presentation  (i.e.,  defense  statement  first  versus 
defense  statement  last)  be  important? 
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RESULTS 


Initial  Responses  of  Technicians 


The  rationale  of  this  study  presented  three  assumptions  about  the  initial  attitudes 
and  opinions  of  the  experienced  technician  population.  Results  pertaining  to  these  assump- 
tions are  presented  below.  Responses  to  each  pre-posttest  item  were  recorded  on  a 60-point 
scale,  with  lower  scores  indicating  greater  acceptance  or  agreement  and  higher  scores  indi- 
cating less  acceptance  or  agreement.  Any  rating  above  30  indicated  overall  disagreement. 


Expectations  of  New  Systems 


The  first  assumption  was  that  the  technician  would  in  general  have  positive  expecta- 
tions of  new  systems.  It  is  recalled  that  the  initial  event  in  this  study  was  an  oral  presentation 
on  three  new  systems.  The  first  three  items  of  the  questionnaire  (Appendix  A,  Page  30) 
asked  the  technicians  whether  they  felt  these  systems:  (1)  would  be  advantageous  to  ship 
operations,  (2)  were  necessary,  and  (3)  would  meet  the  objectives  for  which  they  were 
developed. 

The  mean  ratings  for  these  three  items  were  respectively  7.7,  7.5,  and  16.6,  with 
associated  standard  deviations  of  8.3,  10.4,  and  20.7.  These  data  indicate  that  virtually  all 
technicians  strongly  considered  the  new  systems  to  be  both  advantageous  and  necessary, 
and  most  of  them  also  anticipated  that  they  would  meet  their  intended  objectives. 


General  Evaluations  of  Shipboard  Equipments 

Items  1 through  4 of  the  second  set  of  questionnaire  items  (See  Appendix  A,  page  31 ) 
pertained  to  the  effectiveness  and  competent  use  of  new  equipments  and  operational  equip- 
ments. The  mean  scores  of  these  items  ranged  from  20.6  to  22.1,  with  standard  deviations 
ranging  from  12.2  to  12.8.  These  data  indicate  that  most  technicians  agreed  that  both  new 
and  operational  equipments:  ( 1 ) serve  the  purpose  for  which  they  were  designed,  and  (2) 
are  used  by  technicians  as  intended. 


Causes  of  New  Equipments  Falling  Short  of  Expectations 


Causes  Initially  Accepted  by  Technicians.  The  second  assumption  of  this  study  was 
that  technicians  would  initially  ascribe  problems  with  new  systems  to  causes  which  were 
technical,  fairly  visible,  and  external  to  their  team.  Items  5 through  17  in  the  second  set 
specify  possible  causes  of  new  equipments  falling  short  of  expectations.  Mean  scores  indicate 
overall  agreement  with  8 items  from  this  set  of  13.  These  items  are  listed  in  Table  2 with 
their  associated  means  and  standard  deviations. 

The  means  and  standard  deviations  of  Items  9 and  13  indicate  that  a large  majority 
of  the  technicians  were  in  substantial  agreement  that  problems  with  new  equipments  are  a 
consequence  of  delays  in  receiving  replacement  parts  and  training  inadequacies.  To  a lesser 
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TABLE  2.  DESCRIPTIVE  STATISTICS  ON  EIGHT  ITEMS  WITH 
WHICH  TECHNICIANS  INITIALLY  AGREED. 


Item 

# 

Item 

Mean 

SX>. 

9 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  long 
delays  in  obtaining  replacement  parts  for  the  equipment. 

16.1 

14.6 

13 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  no 
training  was  available  or  the  existing  training  did  not  provide  necessary 
skills  for  adequate  performance. 

16.8 

14.6 

6 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  the 
new  equipment  is  difficult  to  maintain  and  repair. 

22.3 

16.5 

10 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
poor  and  insufficient  documentation  telling  how  to  operate  and  maintain 
the  equipment. 

23.4 

15.9 

14 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  all 
functions  and  modes  of  equipment  were  not  tried  out  under  all  operational 
conditions. 

23.4 

16.9 

5 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
frequent  breakdowns  of  the  equipment. 

25.8 

16.9 

8 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because,  from 
maintenance  technician’s  standpoint,  the  equipment  is  poorly  designed. 

27.6 

14.9 

12 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
insufficient  preventative  maintenance  to  keep  the  equipment  in  working 
order. 

29.4 

15.8 

degree,  most  agreed  that  problems  with  new  equipments  are  caused  by  (1)  difficulties  in 
equipment  maintainability  (Item  6),  (2)  equipment  unreliability  (Item  5),  (3)  insufficient 
operation  and  maintenance  document  (Item  10),  and  (4)  failure  to  try  out  the  equipments 
under  a range  of  operational  conditions  (Item  14).  Design  problems  (Item  8)  and  insuffi- 
cient preventative  maintenance  (Item  12)  appear  to  be  very  mildly  endorsed  as  causes  of 
misuse,  but  the  proximity  of  the  means  to  the  mid-point  and  the  magnitude  of  the  standard 
deviations  indicate  that  many  technicians  did  not  agree  with  these  causes.  Of  the  eight 
items  which  are  summarized  in  Table  2,  only  Item  12,  whose  mean  is  closest  to  30,  presents 
a cause  that  can  be  considered  internal  to  the  technician  team. 


Causes  Initially  Rejected  by  Technicians.  The  mean  score  of  5 of  the  1 3 items  which 
specified  causes  for  equipments  falling  short  of  expectations  were  above  30,  indicating  over- 
all disagreement.  Table  3 presents  these  items  along  with  their  means  and  standard  deviations. 
These  data  indicate  that:  ( 1 ) no  items  were  strongly  rejected  by  most  of  the  technicians, 

(2)  the  4 items  with  the  highest  scores  (least  agreement)  identified  causes  which  were  inter- 
nal to  the  technician  team,  and  (3)  relative  to  the  entire  set  of  13  items,  the  3 which  were 
most  rejected  identified  aspects  of  technicians’  attitudes  as  a cause  of  problems  with  new 
equipments. 
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TABLE  3.  DESCRIPTIVE  STATISTICS  ON  FIVE  ITEMS  WITH  WHICH 
TECHNICIANS  INITIALLY  DISAGREED. 


Item 

# 

Item 

Mean 

S.D. 

7 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because,  from 
an  operator’s  standpoint,  the  equipment  is  poorly  designed. 

31.3 

15.1 

n 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
insufficient  corrective  maintenance  when  the  equipment  breaks  down. 

32.6 

12.9 

15 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  the 
negative  attitudes  of  operators  and  technicians  toward  their  work  in 
general. 

33.0 

17.9 

17 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
officers’  and/or  enlisted  men’s  attitudes  toward  the  specific  equipment 
and  procedures. 

33.3 

17.7 

16 

In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of 
the  negative  attitudes  of  operators  and  technicians  toward  changes  in 
equipment  or  operating  procedures. 

33.7 

17.3 

Rank  Order  of  Seven  Causes  of  Misuse,  Partial  Use,  and  Nonuse  of  New  Equipment. 

The  final  item  of  the  second  set  asked  technicians  to  rank  order  the  following  causes 
of  misuse,  partial  use,  or  nonuse  of  new  systems:  design  of  equipment,  training,  documenta- 
tion, maintenance,  supply,  general  attitudes,  and  specific  attitudes.  General  attitudes  were 
described  as  predispositions  to  reject  new  ways  or  systems  in  general,  while  specific  attitudes 
referred  to  the  rejection  of  specific  modes,  functions,  or  procedures.  The  median  rank  for 
each  cause  is  listed  in  Table  4.  These  data  indicate  that  overall:  (1)  training  deficiencies 
were  considered  the  prime  cause,  (2)  other  causes  which  were  external  to  the  technician 
team  were  given  intermediate  ranks,  and  (3)  the  two  types  of  attitudes  were  ranked  lowest 
as  contributing  to  the  problems  of  misuse. 


TABLE  4.  INITIAL  RANKING  OF  SEVEN  CAUSES  OF  MISUSE, 
PARTIAL  USE,  OR  NONUSE  OF  NEW  SYSTEMS. 


Cause 

Median  Rank 

Training 

1.4 

Maintenance 

3.8 

Design 

3.9 

Supply 

4.1 

Documentation 

4.6 

General  Attitudes 

4.9 

Specific  Attitudes 

5.7 
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Changes  in  Responses  Following  Experimental  Treatments 


Expectations  of  New  Systems 

A repeated  measures  analysis  of  variance  (ANOVA)  was  run  on  each  of  the  first 
three  items.  The  main  effects  were  the  experimental  groups  and  testing  time.  Table  5 pre- 
sents means  and  standard  deviations  over  all  groups  and  ANOVA  summary  data  on  each 
item.  The  data  in  Table  5 indicate  that:  (1)  following  each  of  the  treatment  conditions, 
technicians  continued  to  express  positive  expectations  regarding  the  three  new  systems, 

(2)  on  Items  2 and  3,  there  were  statistically  significant  shifts  toward  less  positive  expec- 
tations, but  the  amount  of  shift  on  the  60-point  scale  was  of  little  practical  consequence, 
and  (3)  a group  by  pre-posttest  administration  interaction  occurred  on  Item  1.  This  inter- 
action is  illustrated  in  Figure  2,  which  shows  that  the  Defense  First  Group  rated  the  new 
systems  as  slightly  more  advantageous  following  the  experimental  manipulation,  while  the 
Control  Group  (ASROC  Observations  Only)  and  Defense  Last  Group  rated  the  new  systems 
as  slightly  less  advantageous. 


TABLE  5.  MEANS  AND  STANDARD  DEVIATIONS  OVER  GROUPS1 
AND  ANOVA  SUMMARY  DATA  FOR  ITEMS  ON  TECHNICIANS’ 
EXPECTATIONS  OF  NEW  SYSTEMS. 


F Values  Obtained  in  ANOVAS 

Item 

Item 

Pretest 

Post  test 

Group 

Pre-post 

Interaction 

# 

Mean 

S.D 

Mean 

S.D. 

(2df) 

(Idf) 

(2df) 

1 

To  what  extent  do  you  think  that  the  equipment 
described  ...  is  advantageous  or  disadvantageous 
to  ship  operations? 

7.7 

7.0 

8.3 

7.2 

.9 

1.0 

3.9* 

2 

Do  you  think  these  new  equipments  described  . . . 
are  necessary? 

7.5 

7.5 

10.4 

8.5 

2.1 

7.8** 

2.0 

3 

To  what  extent  do  you  think  these  equipments 
described  . . . meet  the  objectives  for  which  they 
were  developed? 

16.6 

11.2 

20.7 

7.7 

.3 

6.4* 

.6 

Notes:  1.  Means  and  Standard  Deviations  are  pooled  over  groups  because  there  were  no  significant  group  effects 

on  any  item. 

* Indicates  p <.05 
**  Indicates p <.01 
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Figure  2.  Differential  pre-posttest  responses  to 
Item  1 of  Table  5 by  experimental  groups. 


General  Evaluations  of  Shipboard  Equipments 


The  first  four  items  of  the  second  part  of  the  questionnaire  sampled  general  attitudes 
toward  new  and  operational  systems.  Table  6 presents  means  and  standard  deviations  over 
all  groups  and  ANOVA  summary  data  of  each  item.  The  data  in  Table  6 indicate  that:  ( 1 ) 
following  all  experimental  conditions,  technicians  agreed  less  with  the  statements  that  either 
existent  or  new  equipments  aboard  ship  are  used  as  intended,  (2)  none  of  the  experimental 
conditions  influenced  technicians’  general  agreement  with  the  proposition  that  new  equip- 
ment serves  the  purpose  for  which  it  was  designed,  and  (3)  a group  by  pre-posttest  interaction 
occurred  on  Item  1.  This  interaction,  which  is  illustrated  in  Figure  3,  shows  that  the  level 
of  agreement  was  maintained  by  the  Defense  First  Group  increased  by  the  Defense  Last 
Group,  and  decreased  by  the  Control  Group.  All  groups,  however,  showed  moderate  agree- 
ment on  the  posttest. 


Causes  of  New  Equipments  Falling  Short  of  Expectations 

Items  5 through  17  presented  13  possible  causes  of  new  equipments  falling  short 
of  expectations.  Eight  (Items  5,  6,  7,  8,  9,  10,  13,  and  14)  are  categorized  as  being  external 
and  five  (Items  II,  12,  15,  16,  and  17),  internal  to  the  technician  team. Table  7 presents 
means  and  standard  deviations  over  all  groups  and  ANOVA  summary  data  on  each  item. 

The  data  in  Table  7 indicate  that  the  only  significant  effects  were  pretest-post  test, 
and  that  there  were  significant  changes  on  7 of  the  13  items.  Posttest  results  showed  that 
subjects:  (1)  expressed  less  agreement  with  three  items  (5,  6,  and  9)  and  more  agreement 
with  one  item  (14)  which  presented  causes  considered  external  to  the  technician  team,  and 
(2)  expressed  significantly  greater  agreement  with  all  three  items  (15,  16,  17)  that  referred  to 
technicians’  attitudes  as  causitive  factors.  The  amount  of  change  was  greatest  on  Items  1 5, 

16, and  17. 
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TABLE  6.  MEANS  AND  STANDARD  DEVIATIONS  OVER 
GROUPS1  AND  ANOVA  SUMMARY  DATA  FOR  ITEMS 
ON  GENERAL  EVALUATIONS  OF  SHIPBOARD  EQUIPMENTS. 


F Values  Obtained  in  ANOVAS 

Item 

Item 

Pretest 

Posttest 

Group 

Pre-post 

Interaction 

# 

Mean 

S.D. 

Mean 

S.D. 

(2df) 

(ldo 

(2df) 

i 

In  general,  shipboard  equipment  serves  the 
purpose  for  which  it  was  designed. 

20.7 

12.5 

19.6 

8.7 

.6 

.3 

4.5* 

2 

In  general,  shipboard  equipment  is  used  as 
intended. 

21.9 

12.2 

28.0 

10.9 

.2 

9.3** 

.1 

3 

In  general,  new  equipment  aboard  ship  serves  the 
purpose  for  which  it  was  designed. 

20.6 

12.6 

21.0 

9.5 

.2 

.0 

2.4 

4 

In  general,  new  equipment  aboard  ship  is  used  as 
intended. 

22.1 

12.8 

27.9 

12.5 

2 

8.7** 

.8 

Notes:  1.  Means  and  Standard  Deviations  are  pooled  over  groups  because  there  was  no  significant  group  effects 

on  any  item. 

* Indicates  p <.05 
**  Indicates p <.01 


Figure  3.  Differential  pre-posttest  responses  to  Item  1 
of  Table  6 by  experimental  groups. 
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TABLE  7.  MEANS  AND  STANDARD  DEVIATIONS  OVER  GROUPS1 
AND  ANOVA  SUMMARY  DATA  FOR  ITEMS  ON  CAUSES 
OF  NEW  EQUIPMENTS  FALLING  SHORT  OF  EXPECTATIONS. 


F Values  Obtained  in  ANOVAS 

Item 

Item 

Pretest 

Post  test 

Group 

Pre-post 

Interaction 

# 

Mean 

S.D. 

Mean 

S.D. 

(2df) 

(ldO 

(2df> 

5 

In  the  event  new  equipment  falls  short  of  expecta- 
tions, it  is  because  of  frequent  breakdowns  of 
the  equipment. 

25.8 

16.9 

34.9 

13.7 

.4 

12.2** 

1.1 

6 

In  the  event . . .,  it  is  because,  the  new  equipment  is 
difficult  to  maintain  and  repair. 

22.3 

16.5 

29.4 

14.7 

2.0 

9.8** 

2.5 

7 

In  the  event . . it  is  because,  from  an  operator’s 
standpoint,  the  equipment  is  poorly  designed. 

31.3 

15.1 

29.5 

13.7 

1.9 

.7 

.6 

8 

In  the  event . . .,  it  is  because,  from  maintenance 
technicians’  standpoint,  the  equipment  is 
poorly  designed. 

27.6 

14.9 

28.2 

14.6 

1.2 

.3 

.4 

9 

In  tile  event . . .,  it  is  because  of  long  delays  in 
obtaining  replacement  parts  for  the  equipment. 

16.1 

14.6 

21.1 

16.4 

2.7 

6.4* 

1.3 

10 

In  the  event . . .,  it  is  because  of  poor  and  insufficient 
documentation  tellinghow  to  operate  and  maintain 
the  equipment. 

23.4 

15.9 

23.2 

15.0 

2.4 

.0 

.2 

11 

In  the  event  . . it  is  because  of  insufficient 
corrective  maintenance  when  the  equipment 
breaks  down. 

32.6 

12.9 

30.4 

13.9 

.6 

.6 

.9 

12 

In  the  event . . .,  it  is  because  of  insufficient  pre- 
ventative maintenance  to  keep  the  equipment  in 
working  order. 

29.4 

15.8 

31.2 

14.8 

.3 

.4 

.3 

13 

In  the  event . . .,  it  is  because  no  training  was  avail- 
able or  the  existing  training  did  not  provide 
necessary  skills  for  adequate  performance. 

16.9 

14.6 

18.5 

14.9 

1.6 

.7 

1.4 

14 

In  the  event . . .,  it  is  because  all  functions  and 
modes  of  equipment  were  not  tried  out  under 
ail  operational  conditions. 

23.4 

16.9 

16.6 

13.1 

.1 

7.3** 

.7 

15 

In  die  event . . .,  it  is  because  of  the  negative  atti- 
tudes of  operators  and  technicians  toward  then- 
work  in  general. 

33.0 

17.9 

24.7 

15.0 

1.7 

12.3** 

.2 

16 

In  the  event . . .,  it  is  because  of  the  negative  atti- 
tudes of  operators  and  technicians  toward  changes 
in  equipment  or  operating  procedures. 

33.7 

17.3 

24.5 

14.3 

2.5 

16.4** 

1.7 

17 

In  the  event . . .,  it  is  because  of  officers’  and 
enlisted  men’s  attitudes  toward  the  specific 
equipment  and  procedures. 

33.3 

17.7 

22.6 

14.1 

2.4 

14.3** 

1.1 

Notes:  1 . Means  and  Standard  Deviations  were  pooled  over  groups  because  there  were  no  significant  group  effects 


on  any  item. 

* Indicates  p <.05 
**  Indicates  p <.01 
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Changes  in  Rank  Order  of  Seven  Causes  of  Misuse,  Partial  Use, 

or  Nonuse  of  New  Equipment 

Table  8 compares  pretest-posttest  rankings  over  groups  for  each  of  the  seven  causes 
and  presents  summary  data  from  the  Wilcoxon  Sign  Rank  Tests,  which  assess  the  significance 
of  the  pretest-posttest  changes.  The  results  show  that: 

• There  were  statistically  significant  pretest-posttest  changes  in  the  rankings  of 
three  causes  (maintenance,  supply,  and  specific  attitudes), 

• The  largest  change,  and  the  only  significant  change  in  the  direction  of  a higher 
posttest  ranking,  occurred  in  specific  attitudes,  conforming  to  the  changes  found  on  the 
rating  scales. 

• The  average  technician  ranked  specific  attitudes  lowest  on  the  pretest  and  sec- 
ond highest  on  the  posttest. 

• Technicians  endorsed  training  most  strongly  as  a major  cause  on  both  testing 

times. 

• The  average  technician  rated  maintenance  and  supply  problems  as  less  important 
causes  on  the  posttest  than  on  the  pretest.  However,  this  finding  should  be  tempered  with 
the  recognition  that  when  specific  attitudes  were  rated  higher,  something  else  had  to  be 
rated  lower. 

TABLE  8.  MEDIAN  RANKINGS  OVER  GROUPS  AND 
SUMMARY  DATA  FROM  THE  WILCOXON 
SIGN-RANK  TESTS  ON  PRE-POSTTEST 
RANKINGS  OF  THE  SEVEN  CAUSES. 


Cause 

Ranking 

z Score 

Pretest 

Posttest 

Training 

1.4 

1.6 

1.29 

Maintenance 

3.8 

4.6 

2.81** 

Design 

3.9 

3.6 

- .85 

Supply 

4.1 

5.4 

3.00** 

Documentation 

4.6 

4.8 

.84 

General  Attitudes 

4.9 

4.7 

-1.90 

Specific  Attitudes 

5.7 

3.4 

-3.80** 

Note:  **  Indicates  p <.01 


Debriefing 

The  debrief  questionnaire  contained  six  questions  pertaining  to  changes  in  subjects’ 
responses  from  pretest  to  posttest.  The  questions  and  the  percentages  of  agreement  are 
presented  in  Table  9,  which  shows  the  following: 

• Fewer  Control  Group  members  than  either  defense  group  members  felt  influ- 
enced by  the  ASROC  observations. 

• Relatively  few  members  of  either  the  Defense  First  or  Defense  Last  Groups  felt 
influenced  by  the  defense  statement. 
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TABLE  9.  PERCENTAGE  DISTRIBUTION  OF  RESPONSES  TO 
DEBRIEFING  QUESTIONNAIRE  ITEMS  AND  SUMMARY  OF  FISHERS 
EXACT  TESTS  FOR  DIFFERENCES  BETWEEN  THE 
CONTROL  GROUP  AND  DEFENSE  GROUPS  >. 


Percentages  of  Yes  Response 

Fisher 

Item 

Control 

Def.  First. 

Def.  Last 

Exact 

i. 

Did  the  ASROC  data  cards  influence  you? 

43 

(n=  14)2 

76 

(n=17) 

50 

(n=16) 

p=..U 

2. 

Did  the  defense  statement  influence  you? 

— 

35 

(n=  17) 

13 

(n  = 15) 

p=.123 

3. 

Did  you  recognize  the  final  questionnaire  to  be 
the  same  one  that  you  answered  on  the  first  day 
of  the  experiment? 

100 
(n=  16) 

100 
(n=  17) 

100 

(n=16) 

4 

3a. 

If  so,  did  you  feel  that  you  were  expected  to 
change  your  answers  in  some  way? 

44 

(n  = 16) 

71 

(n  = 17) 

87 

(n  = 15) 

p = .01 

4. 

Do  you  feel  that  your  answers  were  substantially 
different  on  the  final  questionnaire? 

7 

(n  = 16) 

24 

(n=  17) 

19 

(n  = 15) 

p=  .15 

5. 

If  you  changed  your  answers  to  questionnaire  items 
between  the  first  day  and  today,  was  this  basically 
because  your  views  or  attitudes  changed  as  a result 
of  the  ASROC  cards  and/or  the  defense  statements? 

36 

(n=  14) 

69 

(n  = 16) 

73 

(n=  15) 

p=  .02 

Notes:  1.  The  defense  groups  were  pooled  because  of  the  similarity  of  their  responses. 

2.  n’s  vary  somewhat  from  item  to  item  because  some  subjects  failed  to  answer  some  items. 

3.  Fisher  Exact  between  defense  groups  only  because  the  control  group  did  not  respond  to  this  item. 

4.  Fisher  Exact  test  not  conducted  because  of  obvious  absence  of  difference. 


• All  subjects  recognized  the  questionnaire  as  being  the  same  on  both 
administrations. 

• Less  than  half  (44%)  of  Control  Group  members  but  most  members  of  both 
defense  groups  felt  that  they  were  expected  to  change  their  responses  in  some  way. 

• About  one-third  (36%)  of  the  Control  Group  acknowledged  that  any  change  in 
their  responses  was  basically  a result  of  the  experimental  situation,  as  contrasted  with  over 
two-thirds  of  both  defense  groups. 

The  data  in  Table  9 suggest  a possible  subtle  influence  of  the  defense  statement. 

The  only  treatment  difference  between  the  control  and  defense  conditions  was  the  presen- 
tation of  the  defense  statement.  A large  majority  of  the  members  of  the  defense  groups 
denied  that  the  defense  statement  had  any  influence  on  them,  and  all  experimental  groups 
responded  similarly  to  the  questionnaire  items.  Yet,  more  subjects  who  received  the 
defense  statement  felt  that  they  were  expected  to  change,  and  more  of  them  acknowledged 
that  any  changes  were  a consequence  of  the  experimental  conditions. 
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DISCUSSION  AND  CONCLUSIONS 


The  rationale  for  this  study  (pgs.  12  & 13)  stated  five  questions  which  were  of 
primary  concern.  These  questions  pertained  to:  (I)  the  technicians’  initial  assessments  of 
causes  of  misuse  and  nonuse  of  new  equipments,  (2)  the  effect  of  credible  information 
which  attacks  initial  beliefs  (ASROC  observations)  upon  later  assessment  of  causes  of  mis- 
use and  nonuse,  (3)  the  effect  of  the  ASROC  observations  upon  expectations  of  new  systems, 
(4)  the  effect  of  the  ASROC  observations  upon  technicians’  evaluations  of  the  overall  effec- 
tiveness and  utilization  of  new  and  operational  systems,  and,  (5)  the  interaction  of  the 
defense  statement  and  ASROC  observations.  Each  of  these  questions  is  discussed  in  this 
section. 


Technicians’  Initial  Assessments  of  Causes  of  Misuse  and 
Nonuse  of  New  Systems 

A central  assumption  of  this  research  was  that  technicians  would  blame  implementa- 
tion problems  on  causes  which  were  external  to  themselves  or  their  technician  team,  and 
would  reject  internal  causes  such  as  their  attitudes.  The  findings  supported  this  assumption. 
Initially,  the  three  causes  which  were  most  strongly  rejected  referred  to  technician  attitudes, 
while  the  seven  causes  which  were  most  strongly  endorsed  were  external  causes.  Addition- 
ally, in  ranking  seven  listed  causes  of  misuse,  the  two  that  identified  technicians’  attitudes 
received  the  lowest  mean  rank.  It  is  recognized  that  the  attribution  of  external  causes  to 
problems  is  by  no  means  unique  to  Navy  technicians.  Studies  with  other  populations  have 
shown  this  to  be  a more  general  condition  (Jones,  E.  & Nesbitt,  R.,  1971). 


Effect  of  ASROC  Observations  on  Technicians’  Assessment 
of  Causes  of  Implementation  Problems 

As  stated  earlier,  the  ASROC  observations  were  recorded  aboard  ship  during  exer- 
cises. The  observations  provided  sufficient  technical  identifying  information  for  technicians 
to  accept  their  credibility.  Specific  negative  attitudes  toward  new  systems  represented  the 
most  frequent  cause  of  problems.  The  causes  of  problems  were  represented  in  proportion 
to  their  observed  frequency  of  occurrence  in  the  ASROC  study,  and  the  results  indicate 
that  there  were  significant  shifts  in  the  assessments  of  7 of  13  listed  causes  of  equipment 
falling  short  of  expectations.  Changes  were  in  the  anticipated  directions:  all  three  items 
which  identified  technician  attitudes  as  causes  received  greater  posttest  endorsement, 
while  there  was  less  agreement  with  three  of  the  four  items  which  identified  external  causes. 
The  external  cause  item  which  received  significantly  stronger  endorsement  referred  to  effects 
of  insufficient  opportunity  to  try  out  all  functions  and  modes  of  new  equipments  under  all 
operational  conditions.  This  cause  was  cited  in  a few  of  the  ASROC  observations  while 
some  of  the  other  external  causes  were  not.  This  may  have  led  to  the  greater  endorsement. 

It  is  also  noteworthy  that  the  pretest  means  for  each  of  the  three  attitude  items 
were  on  the  “disagree”  side  of  the  scale  (i.e.,  right  side  of  midpoint)  but  the  posttest  means 
were  all  on  the  “agree”  side.  Only  one  of  the  other  four  significant  changes  crossed  the 
midpoint,  and  it  was  an  external  cause  which  went  from  agree  to  disagree. 
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Overall,  the  results  strongly  indicate  that  the  ASROC  observations  selectively 
influenced  technicians’  assessments  of  causes  of  problems  with  new  equipments.  These 
findings  arc  consistent  with  McCroskey’s(  I ‘>72)  conclusion  that  new  evidence  is  influential 
in  producing  immediate  attitude  change.  The  target  altitude  items  were  most  strongly 
affected;  other  items  were  differentially  affected.  Importantly,  technicians  did  not  radi- 
cally discount  external  causes  following  exposure  to  the  ASROC  observations.  They  main- 
tained their  general  acceptance  of  the  external  causes  while  including  their  attitudes  among 
the  causes. 


Effect  of  ASROC  Observations  Upon  Expectations  of  New  Systems 

While  identifying  causes  of  misuse  and  nonuse  of  new  systems,  the  ASROC  obser- 
vations also  presented  the  technician  with  an  intense  reminder  that  new  systems  present 
problems.  It  was  feared  that  such  a message  might  encourage  cynicism  or  other  negative  atti- 
tudes toward  a new  system  that  the  technician  may  be  receiving.  For  example,  the  message 
might  cause  the  Change  Advocate  or  the  technician  team  to  become  overly  negative  toward 
their  new  system.  Obviously  this  effect  would  defeat  the  practical  value  of  the  message,  even 
if  technicians  did  recognize  that  some  of  their  attitudes  could  have  negative  effects  upon  the 
implementation  of  the  new  system. 

The  results  revealed  that  technicians  initially  expressed  positive  expectations  about 
the  three  specific  new  systems  described  at  the  start  of  the  experiment.  Statistically,  post- 
test responses  were  significantly  less  positive  on  two  of  the  three  pertinent  questionnaire 
items.  However,  practically,  the  mean  values  for  all  three  items  were  still  strongly  positive, 
and  a large  majority  of  the  responses  were  still  on  the  positive  side  of  the  scale. 


Effect  of  ASROC  Observations  on  Technicians’  Evaluation  of 
Overall  Effectiveness  of  Hardware  Systems 


To  check  on  other  possible  negative  side  effects  of  the  observations,  respondents 
were  asked  for  overall  evaluations  of  the  effectiveness  and  utilization  of  hardware  sys- 
tems. Once  again,  there  was  concern  that  technicians  might  overreact  to  the  ASROC  observa- 
tions and  generalize  their  dissatisfaction  to  hardware  systems  in  general. 

Posttest  results  revealed  that  respondents  agreed  less  with  statements  that  shipboard 
or  new  equipments  are  used  as  intended.  These  differences  were  statistically  significant, 
and  in  practical  terms  the  change  was  from  moderately  strong  agreement  to  slight  agreement. 
Overall,  respondents  were  not  swayed  from  their  initially  positive  evaluation  that  equip- 
ments serve  the  purpose  lor  which  they  are  designed. 

It  is  difficult  to  determine  whether  the  change  in  evaluations  regarding  the  use  ot 
equipment  represents  an  overreaction  or  a more  accurate  perception  ol  the  real  world.  It  it 
is  an  overreaction,  it  is  noted  that  the  posttest  means  were  still  slightly  on  the  agreement 
side  of  the  scale. 


Effects  of  Combining  the  Defense  Statement  with  the 
ASROC  Observations 

The  defense  statement  was  incorporated  into  this  study  because  it  was  anticipated 
that  the  ASROC  observations  might  present  a message  which  was  too  threatening  and  this 
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might  produce  undesired  effects.  The  results  which  have  been  discussed  reveal  that  the 
ASROC  observations  were  quite  effective  and  were  not  overly  threatening.  In  all  of  the 
major  effects,  the  Control  Group  behaved  similarly  to  the  two  experimental  groups,  sug- 
gesting that  the  defense  statement  had  little  differential  impact. 

Four  statistically  significant  effects  were  reported  in  the  results.  The  first  two  were 
group  by  pretest-posttest  interactions,  which  indicated  that,  relative  to  the  defense  groups, 
the  Control  Group  became  slightly  less  positive  in  (1)  its  anticipation  of  the  effectiveness  of 
the  three  systems  described,  and  (2)  its  agreement  with  the  item  that  shipboard  equipment 
serves  the  purpose  for  which  it  is  designed.  While  both  of  these  interactions  are  somewhat 
as  anticipated  with  respect  to  the  Control  Group,  differences  between  the  defense  groups 
are  not  easily  explicable.  The  fact  that  interactions  were  observed  on  only  2 of  20  items 
suggests  caution  in  placing  meaning  to  these  specific  findings. 

The  second  set  of  statistically  significant  findings  is  perhaps  more  meaningful.  The 
debriefing  questionnaire  contained  six  yes-no  items,  which  pertained  to  subjects’  percep- 
tions of  any  changes  that  had  taken  place.  There  were  no  differences  between  answers  given 
by  the  defense  groups  and  the  ASROC  observation  only  group  on  two  items.  The  first  item 
was  answered  only  by  defense  group  members.  On  the  second  item,  there  was  no  difference 
between  groups  because  all  subjects  in  all  groups  recognized  that  the  same  questionnaire 
had  been  administered  on  two  occasions.  On  the  remaining  four  questions,  fewer  Control 
Group  members  than  defense  group  members  indicated  that  they  had  changed  responses  or 
that  they  were  expected  to  change.  On  two  of  these  four  items,  the  changes  were 
statistically  significant. 

The  interpretation  of  debriefing  findings  is  difficult.  First,  the  preceding  results 
indicate  that  Control  Group  members  changed  their  responses  as  much  as  defense  group 
members  — and  indeed,  all  groups  did  change.  Two  possible  explanations  of  the  apparently 
contradictory  findings  are:  (1)  members  of  the  control  group  were  less  aware  of  their 
changes,  and  (2)  members  of  the  control  group  were  less  willing  to  admit  that  they  had 
changed  their  responses.  If  the  first  explanation  is  correct,  this  would  suggest  that  the  defense 
statement  made  subjects  more  aware  of  the  change  in  attitude  that  was  occurring.  If  the  sec- 
ond is  correct,  it  implies  that  the  defense  statement  provided  a rationale  in  which  one  could 
admit  the  change  more  openly.  In  either  event,  the  major  effect  of  the  defense  statement  was 
to  facilitate  the  subject  to  accurately  report  that  his  responses  had  systematically  changed. 

Since  the  data  did  not  indicate  significant  differences  between  the  two  defense  condi- 
tions, it  is  concluded  that  the  sequence  of  the  defense  statement  relative  to  the  ASROC 
observations  was  of  no  consequence. 


Summary  of  Major  Conclusions 


This  study  was  concerned  with  influencing  technicians’  beliefs  or  attitudes  relative 
to  technical  matters.  Specifically,  an  attempt  was  made  to  enhance  technician’s  awareness 
and  acceptance  of  the  negative  effects  of  some  of  his  own  attitudes.  The  results  of  this  study 
indicate  that  providing  evidence  in  the  form  of  documented  shipboard  observations  is  an 
effective  means  of  communications.  While  the  effect  of  the  communication  may  be  short 
term,  it  will  provide  a necessary  reorientation  for  further  training.  It  is  also  important  to 
note  that  the  effects  of  the  communication  were  specific.  Target  attitudes  were  changed  sub- 
stantially, while  other  attitudes  were  less  influenced.  Finally,  it  was  neither  necessary  nor 
particularly  beneficial  to  amplify  this  evidence  with  a statement  designed  to  provide  context 
and  alleviate  any  threat  which  technicians  may  have  experienced  as  a consequence  of 
exposure  to  the  shipboard  observations. 
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RLCOMMliNDATIONS 


This  study  was  performed  to  provide  input  to  later  stages  of  a broader  research 
effort.  At  this  time,  the  results  are  not  interpreted  to  indicate  any  recommendations  for 
the  operational  Navy.  The  findings  have  been  considered  in  the  construction  of  a Change 
Advocate  training  program. 
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APPENDIX  A 


DATA  GATHERING  QUESTIONNAIRE 


ALL  responses  to  this  questionnaire  are  CONFIDENTIAL.  No  one  other  than  the 
research  staff  will  have  access  to  this  material. 


Name 

Rank 

Years  in  Reserves 

Years  in  Active  Duty 

Time  since  release  from  active  duty:  

years  months 


INSTRUCTIONS:  Please  indicate  the  degree  to  which  you  agree  or  disagree  on  the 
following  scales.  Do  this  by  placing  a line  through  the  scale  at  that  point  that  repre- 
sents your  level  of  agreement. 


SET  1 


1.  To  what  extent  do  you  think  that  the  equipment  described  by  Chief  McCoy 
(BQQ-5,  NVMA,  S3A)  is  advantageous  or  disadvantageous  to  ship  operations? 


advantageous 


disadvantageous 


2.  Do  you  think  these  new  equipments  described  by  Chief  McCoy  (BQQ-5,  NVMA, 
S3A)  are  necessary? 


very  necessary  not  at  all  necessary 


3.  To  what  extent  do  you  think  these  equipments  described  by  Chief  McCoy 
(BQQ-5,  NVMA,  S3A)  meet  the  objectives  for  which  they  were  developed? 


entirely 


not  at  all 


A-2 


SET  2 


1 . In  general,  shipboard  equipment  serves  the  purpose  for  which  it  was  designed. 


agree 


disagree 


In  general,  shipboard  equipment  is  used  as  intended. 


agree 


disagree 


3.  In  general,  new  equipment  aboard  ship  serves  the  purpose  for  which  it  was 
designed. 


agree 


disagree 


4.  In  general,  new  equipment  aboard  ship  is  used  as  intended. 


agree 


disagree 


5.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  frequent 
breakdowns  of  the  equipment. 


agree 


disagree 


6.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  the  new 
equipment  is  difficult  to  maintain  and  repair. 


agree 


disagree 
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7.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because,  from  an 
operator’s  standpoint,  the  equipment  is  poorly  designed. 


agree 


disagree 


8.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because,  from  main- 
tenance technicians’  standpoint,  the  equipment  is  poorly  designed. 


agree 


disagree 


9.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  long 
delays  in  obtaining  replacement  parts  for  the  equipment. 


agree 


disagree 


10.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  poor  and 
insufficient  documentation  telling  how  to  operate  and  maintain  the  equipment. 


agree 


disagree 


11.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  insuffi- 
cient corrective  maintenance  when  the  equipment  breaks  down. 


agree 


disagree 


1 2.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  insuffi- 
cient preventative  maintenance  to  keep  the  equipment  in  working  order. 


agree 


disagree 
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1 3.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  no  training 
was  available  or  the  existing  training  did  not  provide  necessary  skills  for  adequate 
performance. 


agree 


disagree 


14.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  all  functions 
and  modes  of  equipment  were  not  tried  out  under  all  operational  conditions. 


agree 


disagree 


1 5.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  the  nega- 
tive attitudes  of  operators  and  technicians  toward  their  work  in  general. 


agree 


disagree 


16.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  the  nega- 
tive attitudes  of  operators  and  technicians  towards  changes  in  equipment  or  op- 
erating procedures. 


agree 


disagree 


17.  In  the  event  new  equipment  falls  short  of  expectations,  it  is  because  of  officers’ 
and/or  enlisted  men’s  attitudes  toward  the  specific  equipment  and  procedures. 


agree 


disagree 
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1 8.  Please  indicate  in  terms  of  order  of  importance  the  contribution  of  each  of  the 
following  factors  to  the  rate  of  misuse,  partial  use  or  nonuse  of  new  equipment. 
Do  this  by  ranking  these  factors  from  1 to  7 with  1 being  the  most  important 
and  7 witli  I being  the  most  important  and  7 being  the  least  important. 


design  of  equipment 

training 

documentation 

maintenance 

supply 

general  attitudes 

specific  attitudes 
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APPENDIX  B 
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APPENDIX  C 


WRITTEN  DEFENSE  STATEMENT  AND  QUIZ 


The  first  problem  area  we  will  call  design.  One  frequent  evidence  of  a design 
problem  is  that  the  equipment  does  not  perform  to  the  anticipated  standards.  For 
example,  the  range  and  bearing  accuracy  of  solutions  of  a new  fire  control  system  may 
turn  out  to  be  less  accurate  at  sea  than  the  equipment  design  anticipated. 

In  such  situations  we  must  recognize  that  complex  systems  require  the  interaction 
of  many  scientific  disciplines  such  as  physicists,  mathematicians,  and  engineering  psycholo- 
gists; and  communications  problems  sometimes  arise.  Factors  which  optimize  one  type  of 
goal  often  do  so  at  the  expense  of  another.  Sometimes  the  effect  of  these  problems  and 
trade-offs  are  apparent  only  after  the  new  system  reaches  the  fleet. 

It  must  also  be  recognized  that  complex  systems  are  designed  to  operate  in  a 
large  variety  of  situations.  For  example,  systems  which  transmit  or  receive  signals  through 
the  ocean  or  the  atmosphere  must  operate  under  a large  variety  of  water  or  atmospheric 
conditions.  It  is  virtually  impossible  for  a number  of  reasons,  including  cost,  to  simulate 
all  of  these  conditions  in  advance.  Thus,  some  of  the  problems  become  apparent  only 
when  the  system  is  used  in  the  fleet  under  the  specific  conditions. 

If  we  recognize  that  these  types  of  problems  frequently  arise,  we  can  anticipate 
them  and  try  to  help  iron  them  out  as  they  arise.  An  effective  action  when  such  prob- 
lems arise  is  to  feed  the  information  regarding  the  problem  back  to  appropriate  authori- 
ties in  the  Naval  Sea  Systems  Command  and  to  contractor  representatives.  Such  feedback 
can  lead  to  modifications  of  the  system. 

From  a human  engineering  standpoint,  when  design  is  poor,  operator  and  mainte- 
nance tasks  are  made  more  difficult.  These  types  of  problems  have  been  less  evident 
recently,  but  two  causes  of  these  problems  have  been:  (1)  structural  characteristics  of 
the  ship  impose  difficult  constraints  on  equipment  design,  and  (2)  inadequate  knowledge 
of  shipboard  conditions  on  the  part  of  design  engineers.  An  example  of  this  occurred  a 
few  years  ago  when  the  Navy  procured  a dual  trace  oscilloscope  which  was  too  large  to 
carry  to  the  various  compartments  in  which  it  was  to  be  used.  Often  modifications  have 
been  made  to  equipments  after  information  got  back  to  the  appropriate  naval  authorities. 

A second  problem  area  we  will  call  training.  In  the  past,  adequate  training  mater- 
ials typically  have  not  accompanied  new  equipments.  For  example,  equipments  have 
reached  the  fleet  before  either  shore-based  or  shipboard  training  programs  were  available. 
The  Navy  is  now  making  a renewed  and  concerted  effort  to  correct  this  problem;  for 
example,  the  planning  of  training  begins  from  three  to  five  years  before  a new  system 
enters  the  fleet.  The  complexity  of  many  new  systems,  however,  make  it  difficult  to  anti- 
cipate the  full  scope  of  situations  for  which  training  will  be  required. 

There  are  effective  actions,  however,  that  can  be  taken  when  training  deficiencies 
are  encountered.  First,  immediate  statements  of  training  needs  to  type  commanders  can 
result  in  requirements  being  placed  upon  training  commands.  Specific  training  courses 
come  into  being  because  of  stated  fleet  needs.  Once  again,  it  is  necessary  that  the  people 
with  the  problem  communicate  the  problems  effectively  to  the  people  with  the  responsi- 
bility and  authority  to  do  something. 

A third  problem  area  is  maintenance.  Since  World  War  II  we  have  experienced 
the  most  rapid  technological  change  in  the  history  of  man.  New  equipments  have  incorp- 
orated many  state-of-the-art  techniques,  which  were  unknown  five  years  earlier.  Under 
such  conditions  it  can,  and  must,  be  expected  that  on  occasion  some  features  will  have 
bugs  - sometimes  many  bugs.  Just  think  how  many  bugs  often  occur  when  an  automo- 
bile manufacturer  makes  a relatively  minor  change  in  the  design  of  a car  one  year.  These 
bugs  do,  however,  place  a burden  upon  maintenance. 
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As  you  may  know,  many  new  equipments  now  feature  automated  trouble- 
shooting which  should  enable  the  rapid  identification  of  90%  to  95%  of  the  malfunctions, 
which  arc  then  corrected  by  replacing  a card.  While  even  tills  feature  may  still  be  in  the 
debugging  stage,  it  offers  great  hope. 

As  with  other  types  of  problems,  when  information  regarding  recurrent  problems 
is  properly  reported,  it  can  result  in  equipment  modifications. 

Another  type  of  problem  in  the  area  of  maintenance  has  been  logistics.  Time 
required  to  obtain  replacement  parts  has  been  excessive.  The  reasons  for  this  arc  many. 
First,  a large  supply  of  parts  for  a major  system  can  be  very  expensive.  Also,  sometimes 
it  is  difficult  to  anticipate  which  parts  will  be  needed  in  what  quantity.  With  experience 
this  problem  is  often  alleviated,  but  once  again  initial  problems  must  be  dealt  with 
patiently  and  effectively. 

The  last  major  problem  area  we  are  calling  attitudes,  and  there  arc  two  types  of 
attitudes  that  we  will  discuss.  The  first  is  general  attitudes  toward  change  itself,  and  the 
second  is  specific  attitudes  about  features  or  functions  of  the  new  equipment  itself. 

Regarding  general  attitudes,  people  are  often  cither  overly  resistant  to  change  or 
overly  optimistic  about  the  immediate  payoff  of  change.  In  either  case,  a new  system  will 
not  receive  a fair  chance.  The  person  who  seeks  reasons  to  reject  new  systems  will  find 
them.  There  will  be  problems  such  as  those  just  discussed.  The  person  who  is  overly  opti- 
mistic often  becomes  disillusioned  when  problems  arise,  and  he  then  rejects  a potentially 
good  system.  The  necessary  attitude  toward  new  systems  is  to  look  at  each  system  individ- 
ually and  recognize  whom  to  communicate  problems  to  and  how  to  do  it  effectively  in  the 
Navy  setting. 

Regarding  specific  attitudes,  we  find  people  who  dislike  or  resist  specific  features 
or  functions  of  new  systems.  For  example,  many  sonar  technicians  avoided  using  the  PPI 
scope  when  it  was  first  available  on  sonar  sets  because  they  trusted  their  ears  more.  Such 
specific  attitudes  often  result  in  misuse  or  nonuse  of  the  system,  and  a less  than  adequate 
test  of  the  system.  Often  these  attitudes  have  understandably  arisen  because:  (1)  men  have 
not  been  given  adequate  information  about  the  new  feature  or  function,  (2)  the  new  feature 
or  function  has  not  appeared  to  provide  improved  performance  over  the  old  way,  (3)  the 
new  feature  or  function  requires  a new  set  of  skills  and  knowledges,  and  (4)  the  new  feature 
or  function  negatively  effects  or  calls  into  question  the  technicians  status.  Such  specific 
attitudes  must  be  acknowledged.  Technicians  do  not  develop  negative  attitudes  because  they 
like  to.  Once  the  attitudes  and  reasons  for  misuse  or  nonuse  are  understood,  problems  can 
usually  be  corrected.  In  some  cases,  adequate  information  is  all  that  is  required;  in  other 
situations  positive  experiences  with  the  new  feature  or  procedure  will  solve  the  problem. 

In  conclusion,  it  is  essential  that  the  Navy  continually  modernize  and  increase  its 
capabilities.  This  necessitates  change  and  in  some  instances  major  changes.  While  problems 
associated  with  change  cannot  be  completely  avoided,  there  are  avenues  which  can  be 
employed  to  reduce  the  magnitude  of  the  problem. 
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Name 

1.  List  the  four  general  problem  areas  discussed  in  this  presentation,  (pages  37  & 38) 


a. 


b. 


c. 


d. 

2.  List  two  reasons  why  new  equipments  sometimes  do  not  perform  up  to  anticipated 
standards,  (page  37) 


a. 


b. 


3.  List  two  reasons  why  equipment  design  makes  it  difficult  to  operate  or  maintain 
some  new  equipments,  (page  38) 


a. 


b. 


4.  What  is  the  Navy  doing  to  prevent  new  equipments  from  arriving  in  the  fleet  without 
adequate  training?  (page  37) 


5.  List  two  major  reasons  for  maintenance  problems  with  new  equipments,  (page  38) 
a. 


b. 
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2. 


6.  List  two  general  attitudes  that  interfere  with  a new  system  getting  a fair  tryout, 
(page  38) 


a. 


b. 


7.  List  four  reasons  which  account  for  negative  specific  attitudes  about  specific  features 
and  functions  of  new  equipments,  (page  38) 


a. 


b. 


c. 


d. 


8.  Which  of  the  following  statements  best  describes  the  way  that  this  paper  suggests  that 
a technician  should  react  to  problems  with  a new  system?  (pages  37  & 38) 

a.  Be  patient  and  wait  until  the  problem  is  solved. 

b.  Avoid  using  the  system,  or  features  of  the  system  that  cause  problems,  and  the 

Navy  will  see  that  the  system  is  no  good. 

c.  Attempt  to  report  all  problems  through  proper  channels  to  proper  authorities  and 

have  confidence  that  necessary  changes  will  be  made  in  the  future. 

d.  Try  to  solve  the  problems  yourself  and  you  will  be  surprised  how  often  you  will 

succeed. 
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DEBRIEF  QUESTIONNAIRE 


DEBRIEF  QUESTIONNAIRE 


1.  Did  the  ASROC  data  cards  influence  you?  Yes  No 

If  so,  how? 


2.  Did  the  defense  statement  influence  you?  Yes  No 

If  so,  how? 


3.  Did  you  recognize  the  final  questionnaire  to  be  the  same  one  that  you  answered  on 
the  first  day  of  the  experiment? Yes  No 

a.  If  so,  did  you  feel  that  you  were  expected  to  change  your  answers  in  some  way? 

Yes  No 

b.  If  you  thought  that  you  were  expected  to  change  your  answers,  what  do  you 
think  you  were  expected  to  indicate? 


4.  Do  you  feel  that  your  answers  were  substantially  different  on  the  final  questionnaire? 

Yes  No 

5.  If  you  changed  your  answers  to  questionnaire  items  between  the  first  day  and  today, 

was  this  basically  because  your  views  or  attitudes  changed  as  a result  of  the  ASROC 
data  cards  and/or  the  defense  statements?  Yes  No 
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